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ABSTRACT 

There is a lack of knowledge regarding the value and impact of career development 
services, particularly in Canada. The goals of this research were to better understand 
career development services’ evaluation practices and the value of these practices from 
the perspectives of Canadian agencies, practitioners, policy makers, and employers. Data 
were collected using a survey, focus groups, and telephone interviews. Differences were 
found between the type and size of career service providers regarding the importance and 
practice of career development evaluation. Overall, practitioners, agencies, policy mak- 
ers, and employers agreed that it was important to evaluate career development services; 
however, there is a great need to improve how and what is being measured. Implications 
for future research and evaluation practices are considered. 

RESUMfi 

Nous manquons de connaissances sur la valeur et les repercussions des services de perfec- 
tionnement professionnel, en particulier au Canada. Les buts de cette recherche etaient 
de mieux comprendre les pratiques devaluation des services de perfectionnement profes- 
sionnel et la valeur de ces pratiques du point de vue des organismes, praticiens, decideurs 
et employeurs canadiens. Les donnees ont ete recueillies en utilisant une enquete, des 
groupes d’etude et des entrevues telephoniques. Des differences ont ete trouvees selon le 
type et la taille des fournisseurs de services d’orientation concernant l’importance et de 
la pratique de revaluation du perfectionnement professionnel. Dans l’ensemble, prati- 
ciens, organismes, decideurs et employeurs ont convenu qu’il etait important d’evaluer 
les services de perfectionnement professionnel, mais il y a un grand besoin d’ameliorer 
ce qui est mesure et la fa^on de le faire. Des consequences sur la recherche future et les 
pratiques d’ evaluation sont envisagees. 

It is a common contention that the provision of career development services 
facilitates individual career development by improving decision-making, helping 
with the acquisition of satisfactory work, and promoting a balance among mul- 
tiple life roles. Career development services may include individual and group 
interventions and programs, such as career assessment, counselling, information 
and resource management, work development, and community capacity building 
(National Steering Committee, 2004). It is also assumed that career development 
services have the potential to lead to substantial benefits for employers, the labour 
market, and the economy in general (Mayston, 2002). However, there is little 
research providing evidence to support these assumptions about the value and 
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impact of career development services. Furthermore, little is known about the 
evaluation beliefs and practices of those providing career services in Canada. This 
article describes research conducted by the Canadian Research Working Group for 
Evidence-Based Practice in Career Development (CRWG) to better understand 
the current evaluation practices of career development service providers in Canada 
(Magnusson & Lalande, 2005). 

Despite an increasing awareness of the need for better understanding of how 
and why career services are effective (Herr, 2003; Watts, 2005), the number of 
outcome research studies has actually decreased in the last 20 years (Whiston, 
Brecheisen, & Stephens, 2003). The difficulties associated with documenting 
the impact of career interventions may be attributable, in part, to the growing 
recognition of the complexity of career planning and the related challenges of 
efficacy research; (a) the range of factors influencing individual choice; (b) the 
wide variance in client groups, issues, and concerns that makes comparison of 
evidence difficult; and (c) the lack of common outcome measures in the field of 
career development (Hughes, 2004). 

A review of 53 research articles published in the last 10 years in English-lan- 
guage journals on the topic of the efficacy of career development services, high- 
light a number of issues contributing to the lack of evaluation research (Roest 
& Magnusson, 2004). These included the perception of the lack of relevance 
that evaluation holds for practitioners, difficulties in identifying and measuring 
outcomes, and problems in conducting research designs that clearly demonstrate 
cause and effect. Despite these limitations, this review of career efficacy research 
indicated that career interventions or programs (a) had a positive effect on par- 
ticipant satisfaction, (b) increase client exploratory behaviours, and (c) increased 
the likelihood that participants would make career decisions. 

A number of recommendations have been made regarding efficacy research 
of career development services. Roest and Magnusson (2004) suggested that a 
comprehensive research strategy was needed for assessing the efficacy of career 
interventions and taking into account the complexity of variables involved, the 
processes utilized, and the differential and longitudinal impact of interventions. 
Herr (2003) suggests the need for a cost-benefit analysis to document the results 
of career services, and the creation of national research databases to collect and 
distribute such information. Watts (2005) urges efficacy research to link career 
practices to economic efficiency, social equity, and sustainability. In Canada, 
Hiebert (1994) has called for increased and more precise efficacy assessment in 
career counselling. 

Within Canada, two symposia held in 2003 highlighted the need for public 
policy to be guided by evidence pertaining to the efficacy of career development 
practice (Symposia on Career Development and Public Policy, n.d.a). The first 
was an international symposium (Career Guidance and Public Policy: Bridging 
the Gap) with 28 countries represented. The second was a national symposium 
(Working Connections: A Pan-Canadian Symposium on Career Development, 
Lifelong Learning, and Workforce Development) with all provinces and territo- 
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ries represented. Participants included career practitioners, policy makers, and 
employers. A consistent theme that emerged from both symposia was the need 
to develop effective systems for gathering data concerning the impact of career 
development/ career guidance services on a number of levels, such as individual 
well-being, social inclusion, and economic development (Symposia on Career De- 
velopment and Public Policy, n.d.b). Furthermore, participants in both symposia 
discussed the need for data to inform and influence public policy related to the 
provision of career services. They agreed that Canadian career service providers 
are under increasing scrutiny and pressure to prove that the services they offer are 
both cost-effective and beneficial. They also agreed that an important first step 
toward assembling an evidence base for career practice would be to determine the 
current state of outcome evaluation practice in Canada. 

Following these symposia, the CRWG conducted a study to explore the current 
state of practice regarding the evaluation of career development services in Canada 
so as to better understand how evaluation procedures are currently being used, 
whether evaluation is a valued practice, and the nature of the outcomes that career 
service providers were achieving. Specifically, the goals of this research were to: 

1 . Determine what service outcomes are gathered and reported by front-line 
career practitioners and how those outcomes are measured. 

2. Determine the service outcomes gathered and reported at the office, agency, 
or school board levels. 

3. Discover whether there are service outcomes that are being achieved by front- 
line and office, agency, and/or school board levels but are not reported. 

4. Find how policy makers (who fund services) evaluate the services, what they 
want back from the services, and the kinds of evaluation information and 
data they prefer to have about the services. 

5. Determine how employers evaluate career development services in the work- 
place, what outcomes they want from these services, whether the evaluation 
information is useful to them, and the kinds of evaluation information they 
would prefer to receive. 


METHODOLOGY 

The research involved (a) an online survey (in French and English) that was 
made available in the fall of 2004 to Canadian practitioners and agencies that 
provided career development services, (b) focus groups (French and English) 
conducted at the National Consultation on Career Development (NATCON) 
in 2005, and (c) telephone interviews conducted with policy makers and em- 
ployers in December 2004 and January 2005. For the purposes of this research, 
“outcomes” were defined as the specific result or product of an intervention 
including changes in client competence (i.e., knowledge, skills, or attitudes), 
changes in client behaviours (e.g., increased job-seeking behaviours), changes 
in client situation (e.g., employment or educational status), and/or broader 
changes for the client and/or community (e.g., impact on families, community 
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economic development). Interventions were defined as any intentional activity 
implemented in the hopes of fostering client change. The purpose of the survey 
was to explore (a) the importance of assessing the impact of career services, (b) 
how the impact of career services is determined, (c) the results achieved through 
the provision of career services, and (d) the means by which the results were 
measured. The survey included demographic items, rating scales, and open- 
ended questions. 

The purpose of the focus groups was to provide feedback on the themes 
identified in the survey results. After an introduction to the research, the themes 
representing the results of the survey were presented. Participants were asked to 
discuss each theme in terms of whether they agreed or disagreed with the theme, 
adding their own ideas about the themes. 

Telephone interviews were conducted with policy makers and employers. The 
interviews ranged from 10 to 35 minutes in length and included two interviews 
conducted in French. The goal of the interviews was to better understand (a) the 
desired outcomes of career development services, (b) how evaluations are con- 
ducted to determine whether they are obtaining these desired outcomes, and (c) 
the evidence and data they would like to have about these services. 

Participants 

Invitations to participate in the surveys were distributed by e-mail to a master 
list of agencies and practitioners who provide career development services in 
Canada, maintained by the Canadian Career Development Foundation (CCDF). 
This list included participants from Working Connections: A Pan-Canadian Sym- 
posium on Career Development, Lifelong Learning and Workforce Development 
(held in Toronto in November of 2003; n = 150); the Career Circuit Network 
(n = approximately 3000); the Stakeholder Liaison and Advisory Council (SLAC) 
of the Canadian Standards and Guidelines for Career Development Practitioners 
(n = 65); and the Canadian Career Development Foundation Board of Governors 
(n = 1 1). Those receiving an invitation were also encouraged to forward the invita- 
tion to other agencies or individuals engaged in the provision of career develop- 
ment services. A total of 173 agency representatives completed the survey (147 
English-language respondents and 26 French-language respondents). A total of 
214 practitioners completed the survey (168 were Anglophone practitioners and 
46 were Francophone practitioners). 

The participants for the focus groups were recruited at a national conference of 
career services personnel (National Consultation on Career Development) through 
an announcement in the conference program. There were 24 participants in an 
English focus group and 20 participants in a French focus group; practitioners 
and policy makers were represented in both groups. 

The telephone interviews were conducted with 9 policy makers (out of 41 
who were contacted) and 7 employers (out of 23 who were contacted). Letters of 
invitation to participate in a research interview were sent to participants of the 
Canadian symposium, Working Connections: A Pan-Canadian Symposium on 
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Career Development, Lifelong Learning and Workforce Development. Follow- 
up e-mails were then sent to those who did not respond to the initial invitation. 
Individuals who agreed to participate in the research were sent a copy of the ap- 
propriate interview protocol prior to the interview. 

Data Analysis 

Surveys. The relationship between language of respondents and the two global 
criteria variables of importance of outcome measurement and practice of outcome meas- 
urement was assessed using chi-square tests. This was done to determine whether 
the data could be merged for the French and English surveys. Simple frequency 
analyses were conducted for demographic variables, items pertaining to perceived 
importance, and the practices of outcome assessment. Chi-square analyses ( p < 
.05) were conducted between demographic variables and the criteria variables of 
perceived importance of impact assessment and actual practice of impact assess- 
ment. Chi-square analysis was also used to determine whether or not there were 
significant differences on these variables between respondents who completed the 
surveys in English and those who completed them in French. 

The written responses on the surveys were analyzed using content analysis 
techniques, involving the identification of general themes from specific responses 
(Colaizzi, 1978). Twenty participant cases were randomly drawn from the database, 
and verbatim responses were first examined for the presence of themes or pat- 
terns. As each new case was examined, the responses were coded into themes and 
categories; when a new theme or category emerged, each prior case was reviewed 
again to search for the existence of the new theme in that data. This process was 
repeated until no new themes emerged. As a check on the reliability of the themes, 
a second set of 20 cases was randomly drawn. No new themes were found, indicat- 
ing that data saturation had been reached. 

Focus groups. Notes from the focus groups were carefully reviewed. The responses 
were categorized according to (a) accuracy or inaccuracy of the survey results, and 
(b) additional information that was not captured in the survey results. 

Telephone interview data. Written notes from telephone interviews were also 
analyzed using content analysis. The first step in the telephone interview data 
analysis involved reviewing the participants’ responses to each question and group- 
ing similar responses into themes or categories. The number of responses in each 
theme was noted. These initial groupings of responses to each question were then 
reviewed, and higher-order themes were identified that represented categories of 
responses. Because some of the responses to each question were related to some 
of the responses to other questions, the higher-order themes were reviewed across 
the questions. This resulted in two lists of general themes that represented the 
responses made by the group of policy makers and those made by the group of 
employers. 

One individual initially conducted the analysis of the data. Then the original 
responses and themes were sent to another person who confirmed the interpreta- 
tion of the results. This enhanced the validity of the content analysis. 
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RESULTS 

Survey 

A total of 173 agency representatives completed the survey (147 English- 
language respondents and 26 French-language respondents). A total of 214 
practitioners completed the survey ( 1 68 were Anglophone practitioners and 46 
were Francophone practitioners). A chi-square analysis was conducted of the re- 
lationship between language of respondents and the two global criteria variables 
of importance of outcome measurement and practice of outcome measurement. No 
significant differences were found, enabling these data to be merged for subsequent 
analysis of the agency data. 

There was a significant difference between Anglophone and Francophone prac- 
titioners regarding the importance of outcome measurement {~/f = 55.14; p < .01). 
Francophone practitioners were more likely than Anglophone respondents to rate 
the measurement of outcomes as not at all important. There was also a significant 
difference between Anglophone and Francophone practitioners on the practice of 
outcome measurement (yf = 9.39; p < .01). Anglophone practitioners were more 
likely to report on the outcome of their services. Because of the significant differ- 
ences on the key criteria variables of the study, the Anglophone and Francophone 
practitioner data will be reported separately. 

Most respondents from agencies represented not-for-profit agencies (50.3%) 
or provincial government settings (23.3%). Other kinds of agencies represented 
included school (K to 12; 8.6%), post-secondary (8.6%), private (for profit) 
agencies or private practice (6.1%), and federal government agencies (3.1%). The 
agencies reported a wide variety of career development services that they provided. 
The most common services provided by Canadian agencies were career education 
(100%), job search information (94%), career and labour market information 
(89%), individual career counselling (86.7%), employment counselling (83.3%), 
and group counselling (78.6%). 

All client age groups received services provided by both Anglophone and Fran- 
cophone practitioners. A greater percentage of Anglophone practitioners provided 
services to adults, whereas a higher percentage of Francophone respondents provid- 
ed services to younger children. Otherwise, services are similarly distributed across 
age groups. The Anglophone practitioners reported a different pattern of service 
provision from the Francophone practitioners. While all Anglophone practitioners 
reported providing career education programs, they also all reported providing 
assessment services (compared to only 76.2% of the Francophone practitioners). 
There was general agreement between agencies and practitioners about the types 
of career services offered and about the relative ranking of the frequency of the 
provision of these services. However, with the exception of the role of assessment 
for Anglophone practitioners, the rank order of service provision was much the 
same for Francophone practitioners. This distribution of types of services provided, 
as well as the variety of clients served, suggests that this was a representative sample 
of the variety of career development practitioners and agencies in Canada. 
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Participants were asked, “How important is it to measure the outcomes/impact 
of career services?” The responses included (a) not at all important, (b) somewhat 
important, and (c) very important. Agency respondents indicated that it was 
either very important (73.2%) or somewhat important (24.4%). When asked if 
they actually reported on the outcomes or impact of their services, 84% of the 
agencies replied “Yes.” 

Anglophone practitioners agreed on the importance of measuring the outcomes 
of their services (20.5% rated it as somewhat important and 78.9% rated it as very 
important). Forty percent of the Francophone practitioners rated it as somewhat 
important, 33.3% rated it as very important, and 26.7% responded that impact 
measurement was not at all important. The actual practice of impact measure- 
ment for both groups was reported as different in that 71 . 1% of the Anglophone 
practitioners and 46.7% of the Francophone practitioners responded that they 
actually reported on the outcomes or impact of their services. 

A significant relationship was found between the type of service provider and 
the perceived value of measuring the impact of career services for agencies (y 2 = 
25.04; p = .02). Respondents from K-12 school settings were less likely to rate 
the impact measurement as being very important, and those in post-secondary 
settings were more likely to say it was somewhat important. Overall, respondents 
who worked in educational settings placed less importance on impact measurement 
than their counterparts in not-for-profit or government settings. There was also 
a significant relationship between the type of service provider and the perceived 
value of measuring the impact of career services for Anglophone practitioners 
{y 2 = 40.8; p < .01). Practitioners in schools were less likely to view impact meas- 
urement as very important, and not-for-profit agencies were more likely to rate 
it as very important. 

A significant relationship also existed between the type of service provider and 
the actual measurement of service impact for both agencies (y 2 = 32.39; p< .01) 
and practitioners (y 2 = 47.81; p < .01). Agency respondents representing K-12 
settings were less likely to report on the impact of their services; not-for-profit 
agencies were more likely to report on the impact of their services. This finding 
essentially parallels the results noted above for the importance of measuring career 
services impact. Practitioners in federal government agencies, K-12 schools, and 
private practice were less likely to engage in impact reporting. 

Agency respondents reported a significant relationship between the size of the 
service provider and the importance of service impact measurement for agencies 
(y 2 = 24.41; p < .01). In general, the larger the agency, the more importance it 
placed on measuring the impact of its services. No significant relationships were 
found between size of the service provider and the importance of service impact 
measurement for practitioners. When size of service provider was compared to 
whether they reported the actual measurement of outcomes, a significant relation- 
ship was found in the agency data {y 2 = 11.81; p = .01) but, again, not in the 
practitioner data. The larger the agency, the more likely it was to report on the 
outcomes of their services. 
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Measures of association between all remaining demographic variables and the 
criteria variables of perceived importance of outcome measurement and actual report- 
ing of outcomes were also conducted. However, no other significant relationships 
were found. Therefore, it was concluded that, for both agencies and practitioners, 
the kind of services provided, the target age group of clients, the cultural focus of 
services, and the employment status of the intended clients were all unrelated to 
either the importance or the practice of outcome measurement. 

In response to the survey question, “What are the three most important 
outcomes that actually get reported?” practitioners and agencies had similar 
responses. Three themes were identified as being (a) changes in client situation 
such as employment status and goal attainment; (b) the number of clients served; 
and (c) client change, growth, and empowerment. In response to the question 
asking how outcomes were measured, the responses of the practitioners and 
agencies were again very similar. The common means to measure outcomes in- 
cluded (a) frequency counts, (b) interviews and surveys completed by clients and 
stakeholders, (c) observations of client change, and (d) cost-benefit analysis. Ad- 
ditional comments related to the outcomes that were reported and how they were 
measured included themes related to (a) the difficulty of the evaluation process, 
(b) the overemphasis on employment statistics as an outcome, and (c) the need 
for research on the impact of career interventions on special populations. 

Agencies and practitioners responded similarly to the question “What outcomes 
do you believe you are achieving that you are either not required to report or that 
you are not directly measuring?” The common responses were (a) client empow- 
erment, (b) client satisfaction, (c) benefits to the community, and (d) increased 
referrals for services. When asked about the difficulties faced when collecting evi- 
dence to measure the impact of their services, agencies and practitioners seemed to 
agree that some outcomes are complex and unmeasurable. They reported difficulty 
in acquiring client feedback and the absence of agency protocols for evaluation 
procedures. The services are frequently changing, which also makes evaluation 
difficult. Agencies and practitioners also noted that there was a perceived lack of 
importance placed on evaluation and that a lack of agreement existed between 
agencies and funding sources about definitions of outcomes, making it difficult 
to compare service outcomes. In addition, they reported a lack of experience and 
knowledge about how to conduct impact assessments. 

Responses to the question “What things would you like to report on if you 
could?” resulted in similar response themes for agencies and practitioners. These 
responses included (a) social impact of services; (b) client empowerment, growth, 
and satisfaction; (c) service quality and utilization; (d) longitudinal impact of 
services; and (e) how their evaluation results compared with those of other agen- 
cies and national research. 

When asked “How could the value of your services be better measured?” agen- 
cies and practitioners responded similarly by suggesting (a) more consistent service 
delivery goals and measurement methods to assess whether these goals were reached, 
(b) better tools to measure service impact, (c) having evaluation embedded in serv- 
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ice delivery, and (d) increased cooperation between provinces. There was generally 
a reported discrepancy between what practitioners and agencies were required to 
evaluate and what outcomes they believed were more important to report. This led 
to considerable levels of frustration, and a mind set that could be described as “us 
versus them.” For example, practitioners frequently noted that “they” — meaning 
the people responsible for the funding of their services — were not interested in 
anything other than “job placements,” and the implication was that the funding 
sources did not truly appreciate the full impact of the services they were funding. 

Focus Groups 

A total of 21 people attended the English-language focus group and a parallel 
group for French-speaking participants was held at the same time. The participants 
in both groups discussed the findings from the survey and agreed the results pro- 
vided a good representation of how career services impact assessments in Canada 
are valued and conducted. 

Telephone Interviews of Policy Makers and Employers 

Interviews were conducted with nine Canadian career development policy 
developers and seven employers. Policy makers reported that they were interested 
in receiving a range of outcome data from the career development service that 
they fund. They wanted specific outcomes of services, such as client outcomes; 
external indicators of client outcomes; indicators that the services made a differ- 
ence and added a value or return on investment; and longitudinal evidence that 
the outcomes persisted. Policy makers also wanted information about how career 
services were provided (for example, the number of clients and whether the level of 
services were the most appropriate means of service delivery). Current evaluation 
procedures included (a) monitoring financial and activity outcomes; (b) acquiring 
feedback from employers or teachers about client outcomes; (c) use of client port- 
folios; (d) observation of client outcomes; (e) surveys of practitioners, clients, or 
organizations; (f) qualitative data; and (g) acquiring information about whether 
other organizations use the program (e.g., as an indicator of the quality of the 
program). Most policy makers expressed a need to improve the current evaluation 
procedures, and a total of 19 suggestions were made. Examples of these sugges- 
tions include a better understanding of core concepts such as “outcomes,” the 
need to demonstrate cause and effect, better tools for evaluation, and the ability 
to compare outcomes between service providers. 

Employers expressed more interest in tangible results related to their organiza- 
tional effectiveness. As a result of their investment in career development services, 
they wanted trained, motivated, and satisfied employees with increased productiv- 
ity and less employee turnover. Employers were also interested in obtaining feed- 
back from the career development service providers regarding attendance numbers 
and what services seemed to be working best. Three employers said that they were 
getting the information they needed to make informed decisions about the career 
services offered, one disagreed, and two said they needed more information. A 
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variety of suggestions (26 in all) were made regarding how evaluations could bet- 
ter provide the information that employers need. Examples of these suggestions 
include better outcome indicators, employee satisfaction with the services, and 
improved methods for measuring the effectiveness of career development services 
over the long term. 


DISCUSSION 

This preliminary exploration regarding the evaluation of career development 
services in Canada provides some insights into what is currently practiced, valued, 
and desired for practitioners, agencies, policy makers, and employers. The results 
seemed to suggest that Anglophone practitioners valued and practiced the evalua- 
tion of their services more than Francophone practitioners. However, the findings 
also indicated that service providers in the school system (kindergarten to grade 
12) were less likely to evaluate their services than were those providing services 
to adults and in larger agencies. It is important to interpret these two Endings 
together: the Francophone practitioners in this study primarily provided services to 
younger children in educational settings, so the differences in evaluation practices 
and reported values may be more attributable to the type of workplace than the 
language used by the practitioner. The Francophone sample was also very small 
and may not be representative of this population. 

Canadian agencies that deliver career development services agreed that it was 
important to evaluate their services, and most did practice some form of evalua- 
tion. Schools were less likely to report on the impact of their services than were 
not-for-profit agencies. This may be due to the tendency for not-for-profit agencies 
to be accountable to the organizations that fund their services, whereas schools 
tend to be publicly funded in Canada and do not regularly compete for the funding 
of services. It would be interesting to explore the impact that service evaluation 
in educational settings would have on the perceived credibility of those services 
within the educational system. 

Respondents from larger agencies were also more likely to report evaluation 
results than were respondents from smaller agencies. This might be indicative of 
a lack of resources available in smaller organizations to conduct and report the 
evaluation of their services. However, all practitioners and agencies indicated in 
their written responses that evaluation was important and that there is a need for 
better evaluation practices. This was echoed by policy makers who publicly fund 
schools, suggesting that evaluation practices need improvement in all agencies, 
including the school system. 

Overall, the results of this research suggest that in Canada, agency representa- 
tives, career practitioners, policy developers, and employers believe in the impor- 
tance of assessing the impact of career development services. These representatives 
of people who work in the career development community of Canada expressed 
concern about the current state of impact assessment and have many suggestions 
regarding how to improve the evaluation of career development services. 
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Of importance are the discrepancies between how evaluation is currently being 
conducted and reported as compared with the practices and outcomes that re- 
spondents would prefer. The practitioners, agencies, policy makers, and employers 
all expressed dissatisfaction with current evaluation processes and the outcomes 
being measured. All respondent groups indicated that important outcomes were 
being achieved but were neither being reported nor measured in any systematic 
way. This discrepancy led to considerable levels of frustration; however, despite this 
frustration, a broad range of possible outcomes was suggested that could indicate 
the quality of the services provided. A definite interest was also expressed by all 
groups for more standardized definitions of concepts such as outcomes, allowing 
for standardized measurement approaches, and the comparison of results between 
service providers. 

Most of the respondents also expressed the need to develop improved proce- 
dures for measuring outcomes. The nature of the services was described as complex, 
and thus requiring more sophisticated measurement tools and processes for hard- 
to-measure results such as improvement in client confidence. It was also suggested 
that there was a need to develop procedures that can measure the differential 
impact of services and assess the longitudinal impact as well as the immediate 
impact. These evaluation processes and tools were recommended to be integrated 
as core components of service delivery and given the same priority as the services 
provided. A need was identified for more information and training regarding 
how to determine the efficacy of career development services, as well as increased 
resource provision for evaluation from the funding agencies of the services. 

The small sample of employers who participated in this research expressed 
an interest in determining whether career development services were of value to 
their employees. They tended to focus on outcomes that were more relevant in 
their organization and the workplace. These types of outcome indicators may be 
important to include as indicators of how career development services impact the 
labour market and the economy. 

Implications for Research and Practice 

The results of this research need to be considered with caution, as this repre- 
sents an initial exploration of this topic. The survey sample was large; however, 
it is difficult to determine if this is a representative sample of career development 
practitioners and agencies in Canada because there is limited data available about 
this field of practice. The number of respondents who were Francophone, policy 
makers, and employers was also very small. It is difficult to know whether the 
samples of policy makers and employers were representative of people in these 
professions. 

However, the results suggest that there is a demand for more information and 
training regarding effective evaluation procedures that can differentiate between 
the effects of a variety of interventions, determine the impacts over time, and assess 
the cumulative effects of interventions. There is also a need for evaluations that 
can define a variety of standardized outcomes at the individual, organizational, 
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and societal levels. There is a need for more resources for (a) training agencies and 
practitioners regarding evaluation procedures, (b) the development of better evalu- 
ation processes, and (c) increased communication between the stakeholders. 

Future research is needed in the held of evaluation of career development 
services. Clearly defined concepts, inrervenrions, outcomes, and procedures are 
required to allow for the comparison of results across settings and the accumulation 
of knowledge in this area. The demonstration of the efficacy of career development 
services in Canada is needed to confirm the perceived value of these services and 
to increase the efficiency in this important held of practice. 

To fully demonstrate the value of career development services, ir will be impor- 
tant to develop a culture of evaluation for all stakeholders. Such a culture would be 
characterized by the belief that the identihcation of outcomes is an integral part of 
the provision of services. Measuring and reporting outcomes needs to be integrated 
into the provision of career development services, and evaluation practices must 
be included from the initial planning stages through to the actual implementation 
of the services, and beyond to determine the post-intervention effects of services. 
To reach this goal, it will be important for outcome assessment to take a more 
prominent role in counsellor education programs. It will also be important for the 
reporting of outcomes to become a priority for all parties, including practitioners, 
agencies, policy makers, and employers. 

A culture of evaluation will enable career development service providers to 
demonstrate the value of their work to all stakeholders and thereby increase the 
likelihood of receiving support for ongoing practice and improvements in this held. 
Career development services in Canada are recognized as having signihcant value 
and impact for clients, groups, and society; however, further research and develop- 
ment are required to demonstrate these outcomes and inform future practices. 
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